REPORT  DOCUMENTATION  PAGE 


Form  Approved 
OMB  No.  074-0188 


Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and  maintaining 
the  data  needed,  and  completing  and  reviewing  this  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including  suggestions  for 
reducing  this  burden  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302,  and  to  the  Office  of 
Management  and  Budget,  Paperwork  Reduction  Project  (0704-0188),  Washington,  DC  20503  _ _ 


2.  REPORT  DATE  6/16/03  3.  REPORT  TYPE  AND  DATES  COVERED 

Final:  8/15/99-7/31/02 


1.  AGENCY  USE  ONLY  (Leave 
blank) 


4.  TITLE  AND  SUBTITLE 

Nonlinear  Control  and  Identification 


5.  FUNDING  NUMBERS 


6.  AUTHOR(S) 

Munther  A.  Dahleh 
John  N.  Tsitsiklis 


F49620-99- 1-0320 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Massachusetts  Institute  of  Technology 
77  Massachusetts  Ave.  35-402 
Cambridge,  M A  02139 


8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 

6877600 


9.  SPONSORING  /  MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

Air  Force  Aerospace  Research 
Office  of  Scientific  Research 
4015  Wilson  Blvd.,  Room  713 
Arlington,  VA  22203 


10.  SPONSORING  /  MONITORING 
AGENCY  REPORT  NUMBER 

F49620-99- 1-0320 


11.  SUPPLEMENTARY  NOTES 


12a.  DISTRIBUTION  /  AVAILABILITY  STATEMENT 


12b.  DISTRIBUTION  CODE 


1 3.  ABSTRACT  (Maximum  200  Words) 

This  research  is  motivated  by  the  problem  of  motion  planning  of  autonomous  vehicles  in  an 
uncertain  environment.  A  possible  approach  to  reduce  the  computational  complexity  of  the  motion 
planning  problem  for  a  nonlinear,  high  dimensional  system,  is  based  on  a  quantization  of  the  system 
dynamics,  in  the  sense  that  we  restrict  the  feasible  nominal  system  trajectories  to  the  family  of 
time-parametrized  curves  that  can  be  obtained  by  the  Inter-connection  of  appropriately  defined 
primitives.  These  primitives  will  then  constitute  a  ''maneuver  library1 '  from  which  the  nominal 
trajectory  will  be  constructed.  Instead  of  solving  an  optimal  control  problem  over  a  high¬ 
dimensional,  continuous  space,  we  solve  a  mixed  integer  programming  problem,  over  a  much  smaller 
space.  At  the  core  of  the  control  architecture  lies  a  hybrid  automaton,  the  states  of  which 
represent  feasible  trajectory  primitives  for  the  vehicle.  Each  constituent  subsystem  of  the  hybrid 
controller  will  be  the  agent  responsible  for  the  maneuver  execution.  The  task  of  the  automaton  will 
be  the  generation  of  complete,  feasible  and  "optimal'  '  trajectories,  via  the  interconnection  of  the 
available  primitives.  Apart  from  the  reduction  in  computational  complexity,  one  of  the  objectives  of 
this  approach  is  the  ability  to  provide  a  mathematical  foundation  for  generating  a  provably  stable 
hierarchical  system,  and  for  developing  the  tools  to  analyze  robustness  in  the  presence  of 
uncertainty  in  the  process  as  well  as  in  the  environment. 


14.  SUBJECT  TERMS 


15.  NUMBER  OF  PAGES 


Hierarchical  Nonlinear  Control  for  Unmanned  Aerial  Vehicles 

17.  SECURITY  CLASSIFICATION  18.  SECURITY  CLASSIFICATION  19.  SECURITY  CLASSIFICATION 
OF  REPORT  OF  THIS  PAGE  OF  ABSTRACT 


16.  PRICE  CODE 


20.  LIMITATION  OF  ABSTRACT 


NSN  7540-01-280-5500 


Dii  i  Hiiiu  i  iUiV  i  A  £  i  A 

Approved  for  Public  Release 
Distribution  Unlimited 


Standard  Form  298  (Rev.  2-89) 

Prescribed  by  ANSI  Std.  Z39-18 


jOO&cBO\  1 5  I 


HIERARCHICAL  NONLINEAR  CONTROL  FOR  UNMANNED 

AERIAL  VEHICLES 


M.A.  Dahleh 

J.  Tsitsiklis 

Department  of  Electrical  Engineering  and  Computer  Science 
Massachusetts  Institute  of  Technology 
Cambridge,  MA  02139 

AFOSR  F49620-99-1-0320 


1  Summary  of  Past  Research 

Below  is  a  summary  of  the  research  conducted  under  NSF  support,  grant  number  ECS-9907466 
(1999-2003)  and  grant  ECS-9796099  (1997-2000). 

1.1  Identification  of  Complex  Systems 

Here,  we  discuss  the  problem  of  identifying  a  complex  system  with  a  limited-complexity  model 
using  finite  corrupted  data.  Complex  systems  are  ones  that  cannot  be  uniformly  approximated 
by  a  finite  dimensional  space.  Nevertheless,  our  prejudice  is  represented  by  selecting  a  finitely 
parameterized  set  of  models  from  which  an  estimate  of  the  original  system  will  ultimately  be 
drawn.  We  introduced  a  new  formulation  that  shows  how  such  a  model  should  be  selected  from 
data.  We  demonstrated  this  paradigm  on  the  class  of  linear  time- invariant  stable  systems.  We  also 
introduce  a  new  information-theoretic  criterion  for  model  quality  evaluation  that  is  usable  in  the 
case  of  under  modeling. 

1.1.1  Finite  Dimensional  Models  of  Systems  with  Unknown  Dimension 

This  work  addressed  the  fundamental  question  of  identifying  a  finite-dimensional  model  from  data 
generated  by  a  possibly  infinite-dimensional  system  using  noisy  input-output  data.  We  assume 
that  the  actual  process  To  6  T  is  7  away  from  a  given  finitely  parametrized  subspace  Q.  The  main 
result  of  this  work  is  summarized  as  follows.  Let  Go  be  the  best  approximation  of  To  in  Q.  the  the 
following  is  true: 

Theorem  1  Given  the  following  assumption:  T  is  £\,  Q  is  the  set  of  FIR  systems  of  order  M , 
noise  is  white  (or  low  correlated  deterministic),  priori:  dist(To,Q)  is  bounded  by  7,  then  there 
exits  an  input  u  such  that  the  least  squares  estimator  converges  to  Go: 

sup  sup  ||Gn-Go||<(Cl7  +  C2M)^M 

t^w€Wl 

In  addition,  the  estimator  is  not  tuned  to  7.  The  above  convergence  rate  is  also  polynomial. 
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These  results  with  a  complete  discussion  on  input  design  are  in  [11]. 

These  results  were  also  extended  to  classes  of  nonlinear  systems.  In  addition,  the  paper  [12] 
presents  a  consistent  formulation  for  designing  robust  controllers  from  Data.  It  is  shown  that,  if  T 
is  the  standard  Hardy-Sobolev  space,  then  the  resulting  error  bounds  can  be  directly  incorporated 
for  synthesizing  robust  controllers.  All  resulting  problems  can  be  solved  using  convex  optimization 
techniques. 

1.1.2  Model  Description  Complexity  (MDC) 

We  introduced  MDC  as  a  measure  of  distance  between  an  estimate  of  a  model  and  the  actual 
process  in  order  to  evaluate  the  quality  of  a  model  class  for  estimating  a  Data  set  constructed  from 
a  complex  input.  In  the  special  case  where  all  the  model  classes  contain  the  model  that  generated 
the  Data,  MDC  coincides  with  the  minimum  description  length  (MDL)  measure.  Using  MDC,  we 
proposed  a  method  for  order  estimation  when  the  process  has  an  unknown  degree  and  does  not  have 
an  a  priori  known  rate  of  decay.  These  results  are  summarized  in  the  recent  Ph.D  thesis  [14,  15]. 

1.1.3  Surface  Lyapunov  Functions 

Many  systems  of  interest  are  dynamic  systems  whose  behavior  is  determined  by  the  interaction  of 
continuous  and  discrete  dynamics.  These  systems  typically  contain  variables  or  signals  that  take 
values  from  a  continuous  set  and  also  variables  that  take  values  from  a  discrete,  typically  finite 
set.  These  continuous  or  discrete- valued  variables  or  signals  depend  on  independent  variables  such 
as  time,  which  may  also  be  continuous  or  discrete.  Such  systems  are  known  as  Hybrid  Systems. 
Although  widely  used,  not  much  is  known  about  analysis  of  hybrid  systems.  This  work  attempts 
to  take  a  step  forward  in  understanding  and  developing  tools  to  systematically  analyze  certain 
classes  of  hybrid  systems.  In  particular,  it  focuses  on  a  class  of  hybrid  systems  known  as  Piece- 
wise  Linear  Systems  (PLS).  These  are  characterized  by  a  finite  number  of  affine  linear  dynamical 
models  together  with  a  set  of  rules  for  switching  among  these  models.  Even  for  simple  classes 
of  PLS,  very  little  theoretical  results  are  known.  More  precisely,  one  typically  cannot  assess  a 
priori  the  guaranteed  stability,  robustness,  and  performance  properties  of  PLS  designs.  Rather, 
any  such  properties  are  inferred  from  extensive  computer  simulations.  In  other  words,  complete 
and  systematic  analysis  and  design  methodologies  have  yet  to  emerge.  In  this  work,  we  develop 
an  entirely  new  constructive  global  analysis  methodology  for  PLS.  This  methodology  consists  in 
inferring  global  properties  of  PLS  solely  by  studying  their  behavior  at  switching  surfaces  associ¬ 
ated  with  PLS.  The  main  idea  is  to  analyze  impact  maps,  i.e.,  maps  from  one  switching  surface 
to  the  next  switching  surface.  These  maps  are  proven  globally  stable  by  constructing  quadratic 
Lyapunov  functions  on  switching  surfaces.  Impact  maps  are  known  to  be  “unfriendly”  maps  in  the 
sense  that  they  are  highly  nonlinear,  multivalued,  and  not  continuous.  We  found,  however,  that  an 
impact  map  induced  by  an  LTI  flow  between  two  switching  surfaces  can  be  represented  as  a  linear 
transformation  analytically  parametrized  by  a  scalar  function  of  the  state.  Moreover,  level  sets  of 
this  function  are  convex  subsets  of  linear  manifolds.  This  representation  of  impact  maps  allows 
the  search  for  quadratic  Lyapunov  functions  on  switching  surfaces  to  be  done  by  simply  solving 
a  set  of  LMIs.  Global  asymptotic  stability  of  limit  cycles  and  equilibrium  points  of  PLS  can  this 
way  be  efficiently  checked.  The  classes  of  PLS  analyzed  in  this  work  are  LTI  systems  in  feedback 
with  an  hysteresis,  an  on/off  controller,  or  a  saturation.  Although  this  analysis  methodology  yields 
only  sufficient  criteria  of  stability,  it  has  shown  to  be  very  successful  in  globally  analyzing  a  large 
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number  of  examples  with  a  locally  stable  limit  cycle  or  equilibrium  point.  In  fact,  it  is  still  an 
open  problem  whether  there  exists  an  example  with  a  globally  stable  limit  cycle  or  equilibrium 
point  that  could  not  be  successfully  analyzed  with  this  new  methodology.  Examples  analyzed  in¬ 
clude  systems  of  relative  degree  larger  than  one  and  of  high  dimension,  for  which  no  other  analysis 
methodology  could  be  applied.  We  have  shown  that  this  methodology  can  be  efficiently  applied  to 
not  only  globally  analyze  stability  of  limit  cycles  and  equilibrium  points,  but  also  robustness,  and 
performance  of  PLS.  Using  similar  ideas,  performance  of  on/off  systems  in  the  sense  that  bounded 
inputs  generate  bounded  outputs,  can  also  be  checked.  Among  those  on/off  and  saturation  systems 
analyzed  are  systems  with  unstable  nonlinearity  sectors  for  which  classical  methods  like  Popov 
criterion,  Zames-Falb  criterion,  IQCs,  fail  to  analyze.  This  success  in  globally  analyzing  stability, 
robustness,  and  performance  of  certain  classes  of  PLS  has  shown  the  power  of  this  new  methodol¬ 
ogy,  and  suggests  its  potential  towards  the  analysis  of  larger  and  more  complex  PLS.  Results  are 
reported  in  [17,  16,  18,  19] 

1.2  Hierarchical  Nonlinear  Control:  Robust  Hybrid  Automaton 

This  research  is  motivated  by  the  problem  of  motion  planning  of  autonomous  vehicles  in  an  un¬ 
certain  environment.  A  possible  approach  to  reduce  the  computational  complexity  of  the  motion 
planning  problem  for  a  nonlinear,  high  dimensional  system,  is  based  on  a  quantization  of  the  sys¬ 
tem  dynamics,  in  the  sense  that  we  restrict  the  feasible  nominal  system  trajectories  to  the  family 
of  time-parametrized  curves  that  can  be  obtained  by  the  interconnection  of  appropriately  defined 
primitives.  These  primitives  will  then  constitute  a  “maneuver  library”  from  which  the  nominal  tra¬ 
jectory  will  be  constructed.  Instead  of  solving  an  optimal  control  problem  over  a  high-dimensional, 
continuous  space,  we  solve  a  mixed  integer  programming  problem,  over  a  much  smaller  space. 

At  the  core  of  the  control  architecture  lies  a  hybrid  automaton,  the  states  of  which  represent 
feasible  trajectory  primitives  for  the  vehicle.  Each  constituent  subsystem  of  the  hybrid  controller 
will  be  the  agent  responsible  for  the  maneuver  execution.  The  task  of  the  automaton  will  be 
the  generation  of  complete,  feasible  and  “optimal”  trajectories,  via  the  interconnection  of  the 
available  primitives.  Apart  from  the  reduction  in  computational  complexity,  one  of  the  objectives 
of  this  approach  is  the  ability  to  provide  a  mathematical  foundation  for  generating  a  provably 
stable  hierarchical  system,  and  for  developing  the  tools  to  analyze  robustness  in  the  presence  of 
uncertainty  in  the  process  as  well  as  in  the  environment. 

The  proposed  architecture  builds  upon,  and  exploit  the  symmetries  of  autonomous  vehicle 
dynamics,  and  is  based  on  the  definition  of  two  classes  of  primitives:  trim  trajectories  and  “maneu¬ 
vers”  (seen  as  transition  between  two  trim  trajectories).  Fundamental  properties  of  the  proposed 
architecture,  such  as  well-posedness,  reachability,  and  consistency  have  been  characterized. 

The  hybrid  automaton  can  be  seen  as  a  new  modeling  tool ,  in  which  the  ODE  describing  the 
system’s  continuous  dynamics  is  replaced  by  the  transition  rules  on  the  directed  graph  representing 
the  automaton,  and  by  the  associated  hybrid  system  evolution.  Results  on  this  development  are 
reported  in  [4,  5,  6]. 

1.2.1  Application:  Real-Time  Motion  Planning 

A  very  important  problem,  especially  when  controlling  autonomous  vehicles,  is  represented  by 
motion  planning  in  the  presence  of  fixed  or  moving  obstacles.  We  are  using  the  expression  “mo¬ 
tion  planning”  as  opposed  to  the  traditional  “path  planning”  because  we  want  to  emphasize  the 
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role  of  the  dynamics  of  the  system,  or  of  non-holonomicity  constraints,  on  the  allowable  feasible 
trajectories;  this  problem  is  also  known  in  the  literature  as  kinodynamic  planning. 

One  of  the  fundamental  conceptual  steps  in  addressing  a  problem  is  the  selection  of  the  appro¬ 
priate  representation  of  the  system  state,  constraints  and  of  the  decision  variables.  The  approach 
that  we  propose,  through  the  introduction  of  the  hybrid  control  architecture  described  in  the  pre¬ 
vious  section,  can  be  seen  as  a  “maneuver  space”.  Using  this  representation,  the  hybrid  automaton 
already  encodes  all  the  relevant  information  about  the  system  dynamics  and  dynamic  constraints 
(such  as  non-holonomicity). 

The  hybrid  control  architecture  lends  itself  in  a  very  straightforward  manner  to  the  implemen¬ 
tation  of  a  new  class  of  motion  planning  algorithms,  based  on  a  randomized  road  map  approach.  By 
relaxing  the  requirement  for  completeness  to  probabilistic  completeness,  this  class  of  algorithms  can 
ensure  successful  termination  with  arbitrarily  high  probability,  while  maintaining  computational 
tractability.  Results  on  motion  planning  are  reported  in  [3,  7,  8]. 

We  introduced  a  version  of  randomized  roadmap  planner,  which  is  based  on  the  use  of  Lyapunov 
functions  for  local  motion  planning,  and  hence  provides  a  very  efficient  and  natural  way  of  dealing 
with  the  system  dynamics,  with  the  associated  stability  guarantees.  Moreover  the  algorithm  is 
very  general,  and  can  be  used  with  both  continuous  state-space  and  hybrid  models  of  a  vehicle’s 
dynamics. 

From  a  theoretical  point  of  view,  it  was  shown  how  to  perform  uniform  sampling  in  the  reachable 
space  of  the  vehicle,  as  opposed  to  sampling  in  the  input  space.  Real-time  issues  were  directly 
addressed:  in  the  case  in  which  finite  computation  time  and  available  resources  do  not  allow  the 
computation  of  a  feasible  solution  before  a  decision  has  to  be  made,  it  was  shown  how  to  ensure 
safety  and  how  to  choose  likely  candidates  for  further  exploration  [3,  4]. 

Actor-critic  learning  algorithms 

We  have  initiated  a  new  research  direction,  on  actor-critic  algorithms  for  simulation-based  opti¬ 
mization  in  stochastic  control.  These  methods  combine  value  function  approximation  (as  in  Neuro- 
Dynamic  Programming  or  Reinforcement  Learning),  with  gradient  descent  within  a  parametrized 
family  of  policies  or  control  laws.  In  particular,  learning  of  the  value  function  is  used  to  construct 
gradient  estimates  with  improved  statistical  properties.  While  variants  of  actor-critic  algorithms 
have  often  been  proposed  within  the  past  20  years,  it  was  always  on  a  heuristic  basis.  In  our  work, 
we  obtain  for  the  first  time  a  sound  family  of  actor-critic  methods  with  rigorous  convergence  guar¬ 
antees.  Of  course,  much  analysis  remains  to  be  done,  e.g.,  in  demonstrating  some  form  of  statistical 
efficiency  and  in  identifying  promising  applications  [9]. 

Computational  Complexity 

On  this  subject,  we  have  settled  a  long-standing  open  problem.  We  have  shown  that  the  bounded¬ 
ness  of  all  products  of  a  given  finite  set  of  matrices  cannot  be  decided  algorithmically.  This  problem 
relates  to  the  stability  of  time-varying  as  well  as  hybrid  systems.  In  addition,  we  have  completed  a 
systematic  exposition  and  major  survey  of  the  literature  on  the  computational  complexity  aspects 
of  systems  and  control  problems  [1,  2]. 
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1.3  Transitions  to  Technology 

Draper  Laboratory:  The  work  on  autonomous  systems  was  transported  to  Draper  Laboratory 
as  part  of  an  integrated  simulation  and  control  environment  for  autonomous  Helicopter.  With  a 
follow  up  DARPA  support  two  small  Helicopter  at  MIT  and  Draper  have  been  built  to  demonstrate 
this  theory. 
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